Death or survival, which you measure may affect conclusions: A methodological study

Abstract Background and Aims Considering the opposite outcome—for example, survival instead of death—may affect conclusions about which subpopulation benefits more from a treatment or suffers more from an exposure. Methods For case studies on death following COVID‐19 and bankruptcy following melanoma, we compute and interpret the relative risk, odds ratio, and risk difference for different age groups. Since there is no established effect measure or outcome for either study, we redo these analyses for survival and solvency. Results In a case study on COVID‐19 that ignores confounding, the relative risk of death suggested that 40–49‐year‐old Mexicans with COVID‐19 suffered more from their unprepared healthcare system, using Italy's system as a baseline, than their 60–69‐year‐old counterparts. The relative risk of survival and the risk difference suggested the opposite conclusion. A similar phenomenon occurred in a case study on bankruptcy following melanoma treatment. Conclusion To increase transparency around this paradox, researchers reporting one outcome should note if considering the opposite outcome would yield different conclusions. When possible, researchers should also report or estimate underlying risks alongside effect measures.

From patterns in the case studies, we prove a theorem: If the relative risks of survival and death agree as to which subpopulation benefits most from a treatment, then the cumulative hazard ratios, the RD, and the OR will agree with them. This paper is organized as follows: • Ignoring confounding for ease of exposition, Section 2 compares choices of outcome and effect measure in analyzing how age modifies the effect of healthcare system on risk of death from COVID-19.
• Section 3 compares effect measure-outcome combinations in a study on how melanoma and its treatment differently increase different age groups' risks of bankruptcy.
• Section 4 discusses patterns in these two case studies and formalizes them into a theorem proved in the Supporting Information.

| Background
The risk of death or survival in patients with COVID-19 depends heavily on many factors including their age 5 and the relative prevalence of COVID-19 in their healthcare system, relative to that system's capacity. In this case study, we use different effect measures to investigate how the age of patients with COVID-19 modifies the effect their healthcare system has on their risk of death or survival.
For purposes of this case study, we will neglect confounders, that is, mutual causes of COVID-19 mortality and relative prevalence.
Such variables include the rate of testing: Increased testing decreases the measured death rate of COVID-19 by revealing asymptomatic and weakly symptomatic cases. 6 Increased testing also decreases the relative prevalence of COVID-19, since countries with increased testing generally detect COVID-19 outbreaks in time to implement appropriate policy actions to prevent the outbreak from overwhelming their healthcare systems. 7 Thus, increased testing partially "explains away" the strong association between COVID-19 mortality and relative prevalence. Other potential confounders include genetics and lifestyle. We, therefore, intend this case study as an illustrative example of how considering different effect measures, or survival instead of death, may affect conclusions about which subpopulation suffers more from an exposure.

| Effect measures and outcomes
Younger people generally have less risk of dying from COVID-19. While COVID-19 overwhelmed both countries' healthcare systems, it caught Mexico particularly unprepared, 10 at least partially explaining these disparate death rates. Other explanatory variables include Mexico's increased absolute prevalence and accelerated onset of preexisting conditions that increase the risk of death from COVID-19. 10 We will look at how age modifies each of our effect measures.
The relative risk (RR) and the odds ratio (OR) find the disparity between Mexican and Italian 40-49-year-olds more alarming than that disparity among 60-69-year-olds. These effect measures may lead stakeholders to conclude that countries with underprepared healthcare systems should focus on middle-aged patients whose deaths are possible but typically preventable, rather than on older patients who have a substantial chance of dying even if prioritized for treatment.
The other relative risk (RR*), which corresponds to the relative risk for survival, and the risk difference (RD) yield the opposite conclusion.

| Effect measures and outcomes
The RR and the OR suggest that melanoma more sharply increases 80-89-year-old patients' risk of bankruptcy. • In terms of the number needed to treat (1/RD), if we relieved 224 20-34-year-old patients with melanoma of its financial effects, we would expect 1 fewer bankruptcy. In contrast, we would have to relieve an estimated 1053 80-89-year-old patients with melanoma from its financial burden to prevent 1 bankruptcy.

| Discussion
Hospitals often face difficult decisions to stay financially solvent while ensuring that their patients get the care they need. Governments benefit from an understanding of how medical expenses affect citizens' financial stability since they choose which populations to target with interventions such as Medicare. In causal contexts, effectmeasure modification is the study of how a modifier affects the extent to which an exposure causes a disease. In this case study, the modifier is age, the exposure is melanoma, and the disease is  However, this agreement is not substantial: Had 1 more not-ask-risk patient receiving the vaccine contracted symptomatic COVID-19, the two effect measures would disagree.

| Implications
Our theorem may allow researchers to conclude agreement between effect measure-outcome combinations without computing each of them. For example, if RR and RR* agree, then RR and RD automatically agree. However, the potential for near disagreement warns that this agreement may not be substantial. Therefore, we urge researchers to consider and report risks or multiple effect measures when possible.

| How to apply
Our findings are of interest to researchers choosing between effect measures and opposite outcomes and to researchers performing meta-analyses over literature employing varying effect measures and outcome codifications. Our strongest recommendation is that researchers report risks whenever possible. Since no single effect measure or outcome is uniformly superior, we suggest researchers report multiple effect measures, such as the two relative risks, unless there is a standard or purpose-informed choice.
In some fields, there is a standard effect measure-outcome combination. For example, HIV trials defining participants with less than 50 viral RNA copies per milliliter as having reached the measured outcome typically use RD, while trials defining virologic failure as the measured outcome typically use RR or HR. 16 In some studies, the purpose of the study informs the effect measureoutcome choice. For instance, a study recommending a population for prioritized COVID-19 vaccination may employ the RD to save the most lives.
In contexts where there is no clear choice, we recommend that researchers report both relative risks. If they suggest the same conclusion, then our theorem shows that the studied factor also modifies HR, HR*, RD, and OR in the same direction. For example, a meta-analysis of studies testing for RD modification could include a study that showed relative risk modification for each of two opposite outcomes.

| Bivariate delta method
Brumback and Berg 17 suggested the multivariate delta method to test the alternative hypothesis that a factor modifies the RR, RD, and OR in the same direction. This method involves considering a joint distribution with a dimension for each of the three effect measures.
We improve on this recommendation, increasing the strength of the alternative hypothesis and reducing the dimensionality of the applicable joint distribution: We suggest using the bivariate delta method to test the alternative hypothesis that a factor modifies RR, RR*, HR, HR*, RD, and OR in the same direction. By our theorem, it suffices to consider the joint distribution of just the two relative risk ratios and p p p p p p p p 3) . We reject the null hypothesis if the 100(1 -α)% simultaneous confidence region for the relative risk ratios lies completely within the (>1,>1) region or the (<1,<1) region.

CONFLICT OF INTEREST
The authors declare no conflict of interest.

DATA AVAILABILITY STATEMENT
The authors confirm that the data supporting the findings of this study are available within the article and its Supporting Information.

TRANSPARENCY STATEMENT
The lead author Jake Shannin affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.